Exposing omitted moderators: Explaining why effect sizes differ in the social sciences.

Policymakers increasingly rely on behavioral science in response to global challenges, such as climate change or global health crises. But applications of behavioral science face an important problem: Interventions often exert substantially different effects across contexts and individuals. We examine this heterogeneity for different paradigms that underlie many behavioral interventions. We study the paradigms in a series of five preregistered studies across one in-person and 10 online panels, with over 11,000 respondents in total. We find substantial heterogeneity across settings and paradigms, apply techniques for modeling the heterogeneity, and introduce a framework that measures typically omitted moderators. The framework's factors (Fluid Intelligence, Attentiveness, Crystallized Intelligence, and Experience) affect the effectiveness of many text-based interventions, producing different observed effect sizes and explaining variations across samples. Moderators are associated with effect sizes through two paths, with the intensity of the manipulation and with the effect of the manipulation directly. Our results motivate observing these moderators and provide a theoretical and empirical framework for understanding and predicting varying effect sizes in the social sciences.

Structural parameter interdependencies in computational models of cognition.

Computational modeling of cognition allows latent psychological variables to be measured by means of adjustable model parameters. The estimation and interpretation of the parameters are impaired, however, if parameters are strongly intercorrelated within the model. We point out that strong parameter interdependencies are especially likely to emerge in models that combine a subjective value function with a probabilistic choice rule-a common structure in the literature. We trace structural parameter interdependencies between value function and choice rule parameters across several prominent computational models, including models on risky choice (cumulative prospect theory), categorization (the generalized context model), and memory (the SIMPLE model of free recall). Using simulation studies with a generic choice model, we show that the accuracy in parameter estimation is hampered in the presence of high parameter intercorrelations, particularly the ability to detect group differences on the parameters and associations of the parameters with external variables. We demonstrate that these problems can be alleviated by using a different specification of stochasticity in the model, for example, by assuming parameter stochasticity or a constant error term. In addition, application to two empirical data sets of risky choice shows that alleviating parameter interdependencies in this way can lead to different conclusions about the estimated parameters. Our analyses highlight a common but often neglected problem of computational models of cognition and identify ways in which the design and application of such models can be improved. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Empirical comparison of the adjustable spanner and the adaptive toolbox models of choice.

Past research indicates that individuals respond adaptively to contextual factors in multiattribute choice tasks. Yet it remains unclear how this adaptation is cognitively governed. In this article, empirically testable implementations of two prominent competing theoretical frameworks are developed and compared across two multiattribute choice experiments: the adaptive toolbox framework assuming discrete choice strategies and the adjustable spanner framework assuming one comprehensive adaptive strategy. Results from two experiments indicate that in the environments we tested, in which all cue information was presented openly, the toolbox makes better predictions than the adjustable spanner both in- and out-of-sample. Follow-up simulation studies indicate that it is difficult to discriminate the models based on choice outcomes alone but allowed the identification of a small subset of cases where the predictions of both models diverged. Our results suggest that people adapt their decision strategies by flexibly switching between using as little information as possible and use of all of the available information. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Retro-Cue Benefit for Verbal Material and Its Influence on the Probability of Intrusions Under Dual-Task Conditions.

It is well established in the working memory literature, that performance can be improved by cueing attention toward the position of a to-be-tested item, even after that item's presentation. This retro-cue benefit is often characterized as the joint outcome of two different effects: facilitation of recall and memory strengthening at the cued position. While the latter has been mainly explained by increased context-content binding, competing hypotheses exist to explain the facilitation of recall. The present study focuses on two of these hypotheses: the removal of non-cued information and the protection of cued information against interference. I replicate the retro-cue effect for verbal material and provide strong evidence for its protective effect. However, I did not find support for the removal hypothesis. This lack of support follows from two empirical findings: Retro-cueing does not decrease, rather increases the conditional probability of intrusions, and the retro-cue benefit does not interact with memory load.

Hypothesis-Testing Demands Trustworthy Data-A Simulation Approach to Inferential Statistics Advocating the Research Program Strategy.

In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.

Source Memory for Mental Imagery: Influences of the Stimuli's Ease of Imagery.

The present study investigated how ease of imagery influences source monitoring accuracy. Two experiments were conducted in order to examine how ease of imagery influences the probability of source confusions of perceived and imagined completions of natural symmetric shapes. The stimuli consisted of binary pictures of natural objects, namely symmetric pictures of birds, butterflies, insects, and leaves. The ease of imagery (indicating the similarity of the sources) and the discriminability (indicating the similarity of the items) of each stimulus were estimated in a pretest and included as predictors of the memory performance for these stimuli. It was found that confusion of the sources becomes more likely when the imagery process was relatively easy. However, if the different processes of source monitoring-item memory, source memory and guessing biases-are disentangled, both experiments support the assumption that the effect of decreased source memory for easily imagined stimuli is due to decision processes and misinformation at retrieval rather than encoding processes and memory retention. The data were modeled with a Bayesian hierarchical implementation of the one high threshold source monitoring model.